Indonesian Journal of Electrical Engineering and Computer Science 
Vol. 26, No. 2, May 2022, pp. 915~923 
ISSN: 2502-4752, DOI: 10.1159 I/ijeecs.v26.i2.pp9 15-923 m) 915 


Convolution neural network model for fundus photograph 


quality assessment 


Sinan S. Mohammed Sheet!?, Tian-Swee Tan”, Muhammad Amir As’ari?, 


Wan Hazabbah Wan Hitam’, Qi Zhe Ngoo?, Joyce Sia Sin Yin’, Kelvin Ling Chia Hiik? 
'Technical Engineering College of Mosul, Northern Technical University, Mosul, Iraq 
*School of Biomedical Engineering Yin Health Sciences, Faculty of Engineering, Universiti Teknologi Malaysia (UTM), 


Johor Bahru, Malaysia 


3Department of Ophthalmology and Visual Sciences, School of Medical Sciences, Universiti Sains Malaysia (USM), 


Bharu Kelantan, Malaysia 


Article Info 


ABSTRACT 


Article history: 


Received Jan 8, 2022 
Revised Feb 13, 2022 
Accepted Mar 9, 2022 


Keywords: 


Contrast enhancement 

CNN model 

CNN metrics 

Fundus photograph 
Photograph quality assessment 


The excellent quality of color fundus photograph is crucial for the 
ophthalmologist to process the correct diagnosis and for convolutional 
neural network (CNN) models to optimize output classification. As a result 
of main causes as acquire devises efficiency and experience of a physician 
most fundus photographs can have uneven illuminance, blur, and bad 
contrast, in addition to micro-features of retinal diseases, which need to 
force their contrast. Fundus photograph quality assessment method is 
proposed to find out the perfect enhanced color fundus Technique in 
fundoscopy photographs-based CNN model. Five photograph quality 
measurements, in addition to five CNN metrics, were used as standard in this 
study. In this research innovative approach combining photograph quality 
measurement and CNN metrics analysis is proposed to find out the best 
enhance method that is set for the multiclass CNN model. The contrast 
enhancement techniques are evaluated using 267 color fundus photographs 
divided into three retina diseases cases were downloaded from the open- 
source database “FIGSHARE”. The study outcome showed that the 
presented system (single-CNN) can determine well the contrast enhancement 
method, as well as the low-quality fundus photograph then it can boost CNN 
metrics to achieve superior. 
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1. INTRODUCTION 


Fundus photographs are one of the most shared eye ailments photographs that are used for 
diagnosis, identification, and classification of retina illness conditions [1]. Retinal tomography is extensively 
used in the medical judgment of visual illnesses, for instance, diabetic retinopathy, cataract, age-related 
degeneration, and glaucoma. Especially the fundus photographs of diabetic retinopathy are used for 
identifying the onset symptoms of illness conditions such as (hard-exudates, red-lesions, and cotton-wool 
spots) that may lead to blindness [2]. In 2015, 0.4 million patients are partially blind while 2.6 million 
patients suffered from critical vision weakness because of diabetic retinopathy [3]. Furthermore, diabetic 
retinopathy is the main reason for vision loss among patients whose age range is 20-74 years [4] in third 
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world countries such as Malaysia, Indonesia, and Iraq. It has gained the interest of researchers to develop 
different artificial intelligence (AI) models to identify the retinopathy symptoms and classify them accurately 
[5]-[12]. Both ophthalmologists and the AI model require good quality fundus photographs to produce 
accurate diagnosis results. However, several factors could affect the fandus photograph's quality during the 
acquiring process, such as reflection, diffusion, and refraction of light in eye lenses, artifacts of camera noise 
that is caused by the unstable power supply, and movement of the camera [13]. Therefore, it is required to 
preprocess those fundus photographs which include contrast-enhancing to solve the aforementioned problems 
before conducting further analysis. Various contrast enhancement techniques for medical photographs have 
been presented depending on the domain requirement such as median filter, sharpening, and filters-based 
histogram equalization [14]-[19]. Diabetic retinopathy could damage the fine retina vessels, cause bleeding, 
form blood spots on the retina with a variety of sizes along the illness period and eventually cause blurring 
vision. It is crucial to develop a very impressive method to enhance contrast visuality on those fine details to 
help the ophthalmologist or the AI for accurate diagnosis [20], [21]. In this study, many contrast 
enhancement methods have been compared. Their performance in improving the classification accuracy by 
the convolutional neural networks (CNN) was studied. The procedure started with pre-selections of the best 
contrast enhancement methods in the image quality assessments. The enhancement techniques with decent 
performance would be applied to understand their contributions in improving the classification by the CNN 
model. Generally, this study is designed to study the relevance of various contrast enhancements in boosting 
the CNN in classifying the diseased photographs into their respective cluster. 

Levels and low levels. Moreover, pre-processing the contrast between the vessels and the 
background of the retina can boost the segmentation of the vessels in the following stage [22]. Image 
enhancement is a stage to map the distribution of the pixels to a new level to achieve the desired contrast 
level. Nonetheless, not all the enhancement techniques deliver pleasant enhancement. As the accuracy of 
CNN classification of the fundus image strongly depends on the quality of the resulting image, the choice of 
enhancement technique is crucial. 

Histogram equalization is a simple and easy method but tends to over-enhance the image [23]. Pizer 
et al. [24] uses histogram equalization and data augmentation in a CNN-based emotion recognition system 
and achieve an accuracy of 78.52%. An improvement of histogram equalization, namely adaptive histogram 
equalization (AHE), was introduced by Wang et al. [25]. AHE divides the image into specified tiles and 
computes them respectively so that the lightness value of the image can be redistributed evenly at its 
respective tiles. AHE-based hybrid CNN model is widely utilized in digital dental X-ray position 
classification [26]. However, AHE is susceptible to noise amplification, typically in relatively homogeneous 
regions in an image. Pizer later improved the AHE by limiting the amplification, hence it was called contrast- 
limited adaptive histogram equalization (CLAHE) [27]. The implementations of CLAHE in CNN models 
include the classification of mammograms [28], improved facial expression [29], and early detection of skin 
cancer [30]. A novel CLHAE called upgraded CLAHE that is adaptive clip limit-CLAHE (ACL_CLAHE), 
through solving the drawbacks of (CLAHE) which is belong to predefined clip limit value that either cause 
intensive enhancement leads to adding more artifacts or weak enhancement, the author proposed to replace 
the predefine climb limit value with global thresholding dividing by 80 constant [21]. These according to the 
writer shows a big improvement in fine details of fundus photographs and boost the CNN performance better 
than the conventional (CLAHE). 

In 1997, Kim [30] introduced the preservation of the mean brightness of the phtotgraph method, bi- 
histogram equalization (BBHE) that provides a natural look of the image. In 1999, Wang et al. [31]. 
proposed an extension of BBHE, equal area dualistic sub-image histogram equalization (DSIHE), which 
partitions the histogram based on the median threshold value. The minimum mean brightness error Bi- 
histogram equalization (MMBEBHE) attempted to output the enhanced image which gave the lowest 
absolute mean brightness error [32]. However, its application in processing medical photographs remained 
unknown. Turgay proposed to consider the contextual information in transformation curve generation [33]. 
Therefore, a 2-dimensional histogram, which consisted of the original mapping curve and its mutual 
relationship between the targeted pixel and the surrounding pixels, was generated instead of single 
distribution as applied in the other state- of-arts methods. The amount of contextual information relied on the 
window size where more information would be referred to when the bigger window was applied. 

According to Yin et al. [33] introduced Bi-histogram bezier curve contrast enhancement (BBCCE) 
in magnetic resonance imaging (MRI) knee photographs that aims to improve the contrast between the knee 
cartilage and tissue background. Later, Teh et al. [34] introduced an extension of BBCCE, namely prominent 
region of interest contrast enhancement (PROICE), which is an explicitly designed technique to process the 
knee MRI. Therefore, PROICE attempted to improve the distinctiveness of the cartilages instead of the whole 
image. Despite BBCCE and PROICE, several techniques are being introduced to enhance medical 
photographs while retaining the important details in the photographs, such as gamma correction adaptive 
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extreme-level eliminating with weighting distribution (GCAELEWD) [35], tuned single-scale retinex (TSSR) 
[36], and tuned brightness controlled single-scale retinex (TBCSSR) [37]. GCAELEWD was created to 
improve brain CT images by removing the extreme intensity levels, and the degree of augmentation was 
determined by the weight of the intensities. The local HE method was prone to cause blocking effect thus 
required additional interpolation to eliminate the effect between the sub-tile. Meanwhile, TSSR and TBCSSR 
were kernel-based methods that could improve the photographs promptly. The kernels were designed 
experimentally and statistically. These techniques altered the image brightness with a controllable parameter, 
lambda. Generally, a higher lambda value tends to adjust the image to a higher degree, and vice versa. 
However, the author claimed that TBCSSR offered greater flexibility in improving photographs at a higher 
degree of enhancement without causing saturation effect, which could be expected in TSSR. 

According to the results of both the traditional image quality assessment (IQA) and the eye vision 
evaluation, the previous studies showed evidence on enhancing image contrast while also showing weakness. 
However, just because you get a strong quality score with traditional IQA doesn't indicate you'll get better 
computer vision. As a result, unveiling a new picture quality evaluation approach that can suggest the best 
image quality and improve CNN output prediction is critical, as is establishing the best enhancement method 
for the CNN model. To address the flaws in previous research, In comparison to standard IQA, this study 
shows how to employ deep learning to build single-CNN as an image quality evaluation. In addition, test the 
proposed CNN-IQA on a multiclass fundus detection system by identifying which images with poor quality 
should be enhanced and comparing them to traditional full enhancement techniques to show that it can boost 
CNN performance. 


2. METHOD 

The present study proposed a single-CNN model for comparing various contrast enhancement 
methods and the techniques with good achievements in assessments will be applied to improve the fundus 
photographs of three diabetic retinopathy diseases. The enhanced photographs were input into a trained CNN 
model to evaluate their performance and their contributions in classification improvement were identified. 
The main dataset used in this study was collected from the FIGSHARE database that provides photographs 
for the valuation and comparison [36]. A total of 267 samples for three classes (hard-exudates, red-lesions, 
and cotton-wool spots) were collected and then grouped into the respective cluster according to the 
annotation conducted by experienced specialists. Figure 1 shows the overall framework of the proposed 
study. 
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Figure 1. The workflow of the proposed system is identifying the best contrast enhancement techniques to 
assist the CNN model 


2.1. Pre-processing contrast enhancement methods 

A color retinal fundus photographs are commonly influenced by various factors, for instance, the 
blurring effect initiated by the cataract [36], [37], noises caused by electrical switching from neighborhood 
equipment [3]. The selected fundus photographs were enhanced with contrast enhancement techniques [38], 
such as TBCSSR, TSSR, MMBEBHE, CLAHE, BBHE, DSIHE, and 2DHE. The performance of the 
techniques was evaluated with image quality assessments (IQA), such as enhancement measure (EME), peak 
signal-to-noise ratio (PSNR), root mean square error (RMSE), image quality indicator (IQI), and minimum 
absolute error (MAE). The enhancement methods are comprised of TBCSSR, TSSR, BBHE, DSIHE, 
BBCCE, 2DHE, MMBEBHE, CLAHE, and ACL-CLAHE. The result of the compared contrast enhancement 
methods was summarized in Tables 1-5. 
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Table 1. Different contrast enhancement approaches yielded EME values 
Contrast enhancement techniques 


Photographs = TBCSSR TSSR MMBEBHE CLAHE BBHE DSIHE 2DHE 
Normal 4.29 3.15 19.758 17.73 23.27 20.42 28.27 
Cotton-wool spot 3.58 2.13 17.676 15.69 18.14 18.82 27.39 
Hard exudate 3.92 246 12.385 16.62 17.16 13.07 25.43 


Table 2. PSNR (dB) values for various contrast enhancement methods as a result 
Contrast enhancement techniques 

TBCSSR TSSR MMBEBHE CLAHE BBHE DSIHE 2DHE BBCCE 

Normal 20.19 22.48 14.82 24.25 12.76 17.43 13.43 17.84 

Cotton-wool spot 16.331 23.86 15.12 26.58 13.02 16.89 13.27 18.26 

Hard exudate 15.070 27.58 16.25 26.36 13.57 17.26 13.84 18.07 


Samples 


Table 3. RMSE values are achieved by different contrast enhancement techniques 
Contrast enhancement techniques 


Samples — TBCSSR TSSR MMBEBHE CLAHE BBHE DSIHE 2DHE BBCCE 
Normal 127 261 42.13 12.77 21.59 66.62 2.89 0.06 
Cotton-wool spot 127 247 4645 12.47 22.29 67.70 3.86 0.03 
Hard exudate 128 4.24 50.68 22.48 22.38 64.82 3.60 0.03 


Table 4. IQI measurements were obtained using various contrast enhancement techniques 
Contrast enhancement techniques 


Samples _-TBCSSR TSSR MMBEBHE CLAHE BBHE DSIHE 2DHE BBCCE 
Normal 044 049 O71 0.61 044 061 044 0.76 
Cotton-wool spot 0.34 0.50 0.70 0.62 044 0.69 0.45 0.78 
Hard exudate 030 051 0.70 0.62 0.46 0.60 0.46 0.78 


Table 5. MAE values as a result of various contrast enhancement techniques 
Contrast enhancement techniques 

TBCSSR TSSR MMBEBHE CLAHE BBHE DSIHE 2DHE BBCCE 

Normal 18.269 1.058 8.505 1.222 2.061 14.07 0.34 0.012 

Cotton-wool spot 28.099 1.022 9.812 1.253 2.433 15.30 0.42 0.005 

Hard exudate 32.362 1.262 10.511 2.239 2.238 14.25 0.42 0.007 


Samples 


The MATLAB program was used to calculate the data in Tables 1-5. The superior contrast 
enhancement approaches can be identified with their achievements in different assessments, based on the 
value of these methods that have good scores the enhancement approach has been selected. The chosen 
techniques, namely TBCSSR, MMBEBHE, BBCCE, DSIHE, and 2DHE, according to their performance 
derived from Tables 1-5. The ACL-CLAHE, which is a novel filter, is an improved algorithm from the 
conventional CLAHE by replacing the fixed clip limit with an adaptive clipping mechanism that utilizes a 


global threshold method. 


2.2. Enhancement based on single CNN 


In this section, a CNN model was established to classify the diseased fundus photographs, including 
raw photographs and enhanced photographs [39]. The CNN model applied in this study comprised of 
convolution layers with a depth of 4, the kernel filter’s size (3*3), the max-pooling layer with a scope of 2*2. 
The last layer was a fully connected dense layer, which attempted to convert the soft-max activation into 1x6 
vectors in the base model and was finally suited to 1x2 vectors. CNN performance can be evaluated with the 
three metrics, which are output classification accuracy (ACC), precision, sensitivity (Sen), specificity (Spe), 


and F-Score. According to (1)-(5) calculate these CNN metrics [4]: 


ACC = (TntTp) 
(Tyt+Fpt+Tpt+Fn) 
Prec = — 
(Tp+tFp) 
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al E 
Sen = as (3) 
—_ TIN 
SPE = Nr (4) 
Tp 
F — Score = — (5) 
Tp+5(Fpt+Fn) 


Table 6 presented the comparison of single CNN performance values based on using or not the 
enhancement method. besides, it can be seen that using an elective enhancement approach could boost the 
CNN performance. The output value 93% has been chosen as a median value among the values (89, 92, 93, 
95, 89), so the enhancement process will apply to the two classes only (cotton-wool spots, and hard- 
exudates). Finally, the choice to apply enhancements corresponding to weaker classes resulted in a profit for 
them. 


Table 6. Single CNN classification accuracy comparison table 


Dataset class Note:1 for enhanced,0 for raw 
Cotton-wool spots 0 1 0 1 0 1 0 1 
Hard-exudates 0 0 1 1 0 0 1 1 
Red-lesions 0 0 0 0 1 1 1 1 
Average of output classification % 65 98 94 93 1“chosen value 95 89 92 72 


2.3. Transfer learning 

When huge data samples (i.e., photographs) are involved, there will be some parameters in CNN 
layers to be considered [40], [41]. However, small datasets will cause overfitting [40]. To overcome both 
the overfitting problem and fine-tune the pre-trained CNN models, the deep transfer learning concept was 
used. Transfer method can be mathematically expressed as A field and D field, which entails the feature 
zone, X, and peripheral probability, P(X), where X is a sample data point. Thus, the domain can be 
expressed mathematically as D=(X, P(X)). Hence, transfer learning is defined as shown in: 


T = {y, p(V1X)} = {1} (6) 
Y = {y1 Ynbp NEY (7) 


where y is a label space, 7 is the predictive label learned from feature weights vector (xi, yi). 

Undeniably, training a network from scratch was beneficial in terms of their ability to distinguish 
multiclass fundus photographs [42], [43]. Nonetheless, a transfer learning approach has been applied to 
determine the fitting number of classification layers that greatly shorten the training time and allow the 
CNN to attain a strong performance of fundus classification [42], [44], [45]. The deep fully connected layer 
was modified from 1000 classes to classify the photographs into three classes (with diabetic retinopathy 
symptoms) as mentioned above. In this study, For the suggested multiclass retinal diagnostic model, the 
VGG19 network was used. The choice was made due to the previous studies that reported that the VGG19 
net could optimize the classification and detection of diseases from the medical photographs and ECG 
waveform [18], [46], [47]. 


3. RESULTS AND DISCUSSION 

Photograph enhancement is crucial to highlight fine details, eliminate noise and elevate overall 
brightness. These contributions potentially boost CNN performance. The constructed system was thoroughly 
evaluated employing a variety of statistical assessment measures, including output classification accuracy 
(ACC), precision (pre), sensitivity (Sen), and specificity (Spe), as well as the F-factor. The full method is 
employing the contrast models on the fundus of the three classes while the selecting enhancement method is 
on the determined fundus classes. 

Table 7 presented the experimental results obtained by the proposed multiclass model using the pre- 
trained VGG19 net that was fed with the pre-processed dataset, which shows that the classification finding in 
terms of accuracy, sensitivity, specificity, precision, and F-score, despite all classes had been enhanced with 
the best-chosen contrast-enhancing techniques according to results of quality test algorithms but still in range 
of 60%. Finally, using ACL-CLAHE shows the highest CNN metrics overall. 
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Based on the outcomes presented in Table 8, we have also determined the accuracy, sensitivity, 
specificity, precision, and F-score values for the presence of diabetic retinopathy (cotton-wool spots, red- 
lesions, and hard-exudates), and they are found to be 80% for TBCSSR and 90.3% for both DSIHE and 
BBCCE, respectively. Furthermore, 93.1% for 2DHE, and 100% for both MMBEBHE and ACL-CLAHE. It 
is expected that the presented scheme would produce the best results if it tested using photographs enhanced 
according to the comparison enhance Table 6, in another word the contrast enhancement procedure will apply 
only for the dataset that proves their needing for enhancement based on single CNN performance. To 
evaluate the performance of the produced system and allow the user to engage with its features, a simulation 
was conducted. 


Table 7. CNN metrics with applying enhancing process on the three classes (full enhance) 


Enhancement model ACC Pre Sen Spe F-score 
TBCSSR 33.33% 51.11% 25.83% 67.66% 28.72% 
DSIHE 45.83% 53.70% 45.08% 73.11% 45.44% 
BBCCE 51.39% 51.85% 51.38% 75.78% 50.59% 
MMBEBHE 54.55% 59.09% 54.13% 77.36% 54.17% 
2DHE 45.83% 52.78% 46.66% 72.72% 46.21% 
CLAHE 57.58% 60.10% 56.85% 79.02% 56.77% 
ACL_CLAHE 60.61% 65.66% 60.56% 80.33% 60.52% 


Table 8. CNN metrics using dataset enhanced based on selecting method (only the class with fair quality had 
been enhanced) 


Enhancement model ACC. Pre. Sen. Spe. F-score. 
TBCSSR 80.00% 80.00% 80.56% 90.24% 79.80% 
DSIHE 90.28% 90.28% 90.65% 95.25% 90.24% 
BBCCE 90.28% 90.28% 92.47% 95.76% 90.07% 
MMBEBHE 100% 100% 100% 100% 100% 
2DHE 93.06% 93.06% 93.10% 96.54% 93.05% 
CLAHE 100% 100% 100% 100% 100% 
ACL-CLAHE 100% 100% 100% 100% 100% 


Furthermore, as shown in Table 8, the superior results were for MMBEBHE, CLAHE, ACL- 
CLAHE, and 2DHE, demonstrating that contrast enhancement algorithms based on histogram equalization 
performed better than the other contrast enhancement approaches. In addition, Table 8 shows that 
MMBEBHE, CLAHE, and ACL-CLAHE achieved the highest performance values equal to 100%. Finally, 
Table 7 reveals that for the ACL-CLAHE CNN measure, TBCSSR performance is the best using the full 
enhance method (the entire dataset has been improved), however, Table 8 shows that the TBCSSR enhancing 
strategy failed to improve CNN performance when using the select enhance method (only the selected classes 
had been enhanced). 


4. CONCLUSION 

Algorithms using the convolution neural network model for fundus photograph quality detection and 
contrast enhancement of diabetic retinopathy for the three cases (hard-exudates, red-lesions, and cotton-wool 
spots) from color fundus photographs have been proposed. A decision-support model for choosing the best 
contrast-enhancing technique with determining who is the disease class to enhance has been designed based 
on the quality of output photograph and single CNN performance. Finally, a contrast enhancement method 
derived from histogram equalization proved their excellent ability to boost the CNN performance compared 
with other methods. Moreover, the improved CLAHE filter achieved superior results in improving the 
performance multiclass CNN system, compared with previous procedures of contrast-enhancing color fundus 
photographs. The presented system can further be employed to additional medical photographs with different 
acquired methods. 
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